When receiving messages, blobs will be deduplicated with the new function create_and_deduplicate_from_bytes(). For sending files, this adds a new function set_file_and_deduplicate() instead of deduplicating by default.

This is for #6265; read the issue description there for more details.

TODO:

• Set files as read-only
• Don't do a write when the file is already identical
• The first 32 chars or so of the 64-character hash are enough. I calculated that if 10b people (i.e. all of humanity) use DC, and each of them has 200k distinct blob files (I have 4k in my day-to-day account), and we used 20 chars, then the expected value for the number of name collisions would be ~0.0002 (and the probability that there is a least one name collision is lower than that) ¹. I added 12 more characters to be on the super safe side, but this wouldn't be necessary and I could also make it 20 instead of 32.
  • Not 100% sure whether that's necessary at all - it would mainly be necessary if we might hit a length limit on some file systems (the blobdir is usually sth like accounts/2ff9fc096d2f46b6832b24a1ed99c0d6/dc.db-blobs (53 chars), plus 64 chars for the filename would be 117).
• "touch" the files to prevent them from being deleted
• TODOs in the code

For later PRs:

• Replace BlobObject::create(…) with BlobObject::create_and_deduplicate(…) in order to deduplicate everytime core creates a file
• Modify JsonRPC to deduplicate blob files
• Possibly rename BlobObject.name to BlobObject.file in order to prevent confusion (because name usually means "user-visible-name", not "name of the file on disk").